Click any tag below to further narrow down your results
Links
OpenAI’s data productivity team built Kepler, an AI agent that understands their data platform context and writes iterative SQL to answer complex data questions. It handles table discovery, query refinement, follow-up prompts and even visualizations, cutting analysis time and reducing errors.
- OpenAI built Kepler, an internal AI agent that writes iterative SQL to answer data questions across 70,000 datasets and 600 petabytes daily, deployed via Slack, IDEs, or web UI
- It doesn't just query once—it discovers tables, refines CTEs, verifies numbers against dashboards/Notion, and proposes hypotheses (e.g., duplicate logging) for anomalies like a WAU spike
- Memory for self-learning plus rigorous evals let it improve over time without regressing
- Rolled out to 80% of the company, turning multi-engineer, multi-hour investigations into minutes
The author shares their shift from using Excel and Google Sheets to DuckDB and SQL for handling CSV files, highlighting the efficiency of querying data directly. They discuss the benefits of using SQL for data manipulation and invite readers to share their own CSV handling tips.
- DuckDB lets you run SQL directly on CSV files (joins, window functions, dedup queries) without importing them into Excel/Sheets first
- You can query CSVs from URLs or cloud storage like S3 directly, and export results straight back to CSV
- Several users find SQL queries more intuitive than replicating the same logic with spreadsheet formulas or pivot tables
- Some commenters mix in Parquet files and other extensions alongside DuckDB to further speed up ad-hoc analysis